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ABSTRACT 

The Model My Watershed (MMW) application, and associated curricula, provides students with meaningful opportunities to 
connect conceptual understanding of watersheds to real-world decision making. The application uses an authentic hydrologic 
model, TR-55 (developed by the U.S. Natural Resources Conservation Service), and real data applied in selected communities 
in southeastern Pennsylvania. The study investigates whether MMW is an effective tool for increasing students' understanding 
of watersheds and the impact of human decisions on local watershed conditions. While statistically significant learning gains 
were measured, most students failed to reach the highest levels of watershed understanding. Further refinement of the MMW 
curricula is needed to help students trace water along multiple pathways and to enable them to trace the connections among 
groundwater, surface water, and atmospheric water vapor. © 2014 National Association of Geoscience Teachers. [DOI: 10.5408/ 
12-395.1] 
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INTRODUCTION 

For most of human existence, we, like other organisms, 
have been subjected to the natural variations that have 
typified the evolution of Earth's environment. However, in 
the last half century, we are no longer just objects affected by 
our changing planet; we are now major agents of that 
change (Williams, 2005). Through the extraction of materials 
to support industrial production, the building of cities to 
house our growing population, and the feeding of multi¬ 
tudes, we have altered Earth's surface. We are no longer 
simply passengers but are now, truly, engineers on 
Spaceship Earth. Furthermore, our identities are often linked 
to the places where our societies have evolved and where 
our cultures have imbued meaning into the places we call 
home. As planetary engineers, it behooves us to understand 
Earth dynamics and to evaluate the potential impacts of our 
actions as part of our decision-making process. As social 
beings, we must also respect indigenous knowledge and link 
natural conditions to cultural understanding of the places we 
occupy. This understanding and evaluation must begin in K- 
12 classrooms (BOSE, 2011, p. ES-1). 

Students should be given problems—at levels appropri¬ 
ate to their maturity—that require them to decide what 
evidence is relevant and to offer their own interpretations of 
what the evidence means. This puts a premium, just as all 
science does, on careful observation and thoughtful analysis. 
Students need guidance, encouragement, and practice in 
collecting, sorting, and analyzing evidence, and in building 
arguments based on it. However, if such activities are not to 
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be destructively boring, they must lead to some intellectually 
satisfying payoff that students care about (AAAS, 1989, p. 
148). 


THE MODEL MY WATERSHED APPLICATION 

Meaningful learning about our changing Earth requires 
a pedagogy that provides opportunities for the learner to 
explore their role as planetary engineers in an authentic way. 
Place-based projects where students investigate their own 
surroundings provide such authenticity. Using students' 
"home turf" as the object of exploration provides context 
and relevance that enhance engagement and promote 
meaningful learning (Powers and Duffin, 2004; Semken 
and Butler-Freeman, 2008). Having a "place" is the first step; 
having "tools" to explore that place is the second step. 
Model My Watershed (MMW) (NSF DRL ITEST #0929639; 
http://wikiwatershed.org/model.html) is a visually engaging, 
free, Web-based application, currently available for areas in 
southeastern Pennsylvania. It invites students to explore and 
evaluate the health of their local watershed. 

MMW is a Web-hosted application that is freely 
accessible through all major Web browsers. It allows 
students the ability to explore, model, and modify their 
local watershed using an authentic model and real data, thus 
providing them with the opportunity to draw scientifically 
valid conclusions. Upon entering the MMW Web site, 
students are greeted with a familiar interface. Built on a 
Google" base, the MMW Web application allows students 
the ability to locate their home simply by searching their 
address. The geographic information systems (GIS) program 
then displays the watershed boundaries and gives students 
the ability to manipulate data with a click of a mouse. (While 
currently only available for southeastern Pennsylvania and 
northern Delaware, the application has the capacity to be 
expanded to include all areas of the globe.) Within the 
MMW Web site, students can explore a variety of data from a 
single map as the integrated data layers, which can be 
toggled on and off, appear as overlays on the map. In 
addition to land-cover information, these data include: water 
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FIGURE 1: The Simple Hydrologic Model allows students to manipulate variables, such as storm intensity, land- 
cover type, and soil texture, and to view the water budget, which partitions the rainfall into evapotranspiration, 
runoff, and infiltration. 


bodies, municipal boundaries, school districts, legislative 
districts, and environmental justice areas, as defined by the 
U.S. Environmental Protection Agency (2013). In addition, 
elevation and soil-group data appear as averages for a given 
location. While having multiple streams of data from 
different government agencies accessible on a single map 
is exciting, the truly innovative feature of the MMW portal is 
its ability to "model" the local hydrology. The application 
has incorporated TR-55, the hydrologic model used by the 
National Resources Conservation Service (NRCS), and it 
integrates enhanced curve numbers for urbanized areas, 
based on the research of Robert Pitt (1999). The data that 
support the modeling effort are from the National Land 
Cover Database, the United States Department of Agricul¬ 
ture (USDA) Soil Survey, and the United States Geological 
Survey (USGS) elevation data set, from which slope is 
derived. In addition, the model incorporates climate 
constants for summer temperature and relative humidity, 
thus allowing it to calculate regional evapotranspiration (ET). 
The model does not calculate peak flows or flow paths, but 
rather it partitions precipitation into a water budget, where 


the distributions of infiltration, runoff, and evapotranspira¬ 
tion are compared to pre-Columbian conditions. 

The MMW web-hosted application consists of four 
toolsets: The Simple Hydrologic Model, Show My Water¬ 
shed, Modify My Watershed, and The Teacher Interface. 

To introduce users to the basic concepts of the model, 
MMW includes the Simple Hydrologic Model, which allows 
users to manipulate the amount of rainfall, the land-cover 
type, and the soil texture irrespective of place (Fig. 1). By 
comparing various combinations, a user can see the impact 
to the water budget of different combinations of land-cover 
type, soil texture, and rainfall intensities. 

The Show My Watershed Module (Fig. 2) places users in 
the landscape and allows them to select a location to 
investigate in one of several ways. They can search for a 
specific address, or select various levels of USGS HUC 
(Hydrologic Unit Code) basins. They can also designate a 
flow point, and the model will identify the area that drains 
water through that point. Finally, users can free-draw an 
area on the map. Once the user has defined an analysis area, 
the model will display the existing land cover and calculate 
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FIGURE 2: The Show My Watershed module allows students to identify an area on the map by identifying a USGS 
HUC Basin, defining a flow point, or drawing an area. Once the area is defined, the student can run the model and 
compare the water budget for the current land use to the water budget during pre-Columbian conditions. 


both the area and percentage in each land-use category. 
When the user then chooses a storm intensity, depicted in 
inches of rainfall, and activates the "start simulation" button, 
the model runs the simulation and returns a water budget 
that depicts infiltration, runoff, and ET for that area based on 
current land-cover type and the storm magnitude that the 


user selected. This water budget is compared to one based 
on pre-Columbian conditions, the U.S. EPA's target for site 
hydrology. 

In the Modify My Watershed Module (Fig. 3), the user 
can make changes to the area previously defined in the 
Show module. S/he can change the land-cover type to any 



FIGURE 3: Once the student has defined an area and modeled it in the Show My Watershed module, s/he can then 
modify the area by changing the land-cover type or implementing best-management practices such as green roofs, 
vegetated infiltration basins, and no-till farming. 
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that is depicted in the model or can implement selected best- 
management practices, such as green roofs, vegetated 
infiltration basins, or porous pavement. S/he can also elect 
to make changes that average improvements over larger 
areas, such as the implementation of rain gardens or cluster 
housing. The model uses an algorithm to integrate those 
changes within the area defined by the user (e.g., 10% of 
homes have rain gardens, or houses are clustered to provide 
the number of housing units equal to high density but open 
space equal to medium density). The resulting output shows 
before and after modifications in the water budgets. It also 
calculates a hydrology score, a water quality score, and a 
social impact score, based on the perceived human 
disruption caused by the change, as ± percentage scores. 
Finally, it keeps track of the approximate financial costs 
associated with each modification. By trying to keep a 
reasonable balance on costs, as well as social and 
environmental impacts, users can weigh the various options 
and make informed decisions on trade-offs that are 
necessary when seeking solutions to complex problems. 
(The underlying formulas and metrics used to calculate those 
scores are available via the teacher interface. Teachers can 
utilize this information in classroom instruction.) 

MMW also includes a Teacher Interface, where teachers 
can enroll students by section, assign scenarios, and track 
time-on-task. Teachers can also develop their own scenarios 
and curricula and share them with peers, if they wish. For 
example, they can challenge students, either individually or 
in teams, to improve the hydrology of their schoolyards or 
try to minimize the environmental impact of a local 
shopping center. The teacher can define explicit goals and 
set a budget as well. Once students have completed their 
assignments, the teacher can export results to an Excel 
spreadsheet for grading purposes. 

PROJECT-BASED LEARNING APPROACH 

As stated already, every child lives in a watershed, and 
every child is Earth's essential steward. MMW is unique in 
that it allows the student to vividly see the impact of human 
action on his/her watershed. It also permits students to 
engage in land-planning exercises that allow them to 
immediately see the potential impact of new changes to 
their watershed. By linking processes that act at the local, 
familiar scale to conditions that occur at a larger scale, the 
MMW Web application seeks to provide participants with a 
cognitive anchor that will allow them to link their own 
observations to data and to the models, thus helping 
learners to comprehend the integrated and complex nature 
of environmental science. 

This project is based on broad problem-based learning 
(PBL) models. The central tenet of PBL methodology is that 
information is mastered when it is taught in the context in 
which it will be used (Donner and Bickley, 1993). "A project 
is meaningful if it fulfills two criteria. First, students must 
perceive the work as personally meaningful, as a task that 
matters and one that they want to do well. Second, a 
meaningful project fulfills an educational purpose. Well- 
designed and well-implemented project-based learning is 
meaningful in both ways" (Larmer and Mergendoller, 2010, 
p. 34). In MMW, we promote the use of scenarios that 
describe a local watershed problem as a vehicle to move 
students from the theoretical application of ideas to active 


problem solving (Farley et al., 2005; Beddoe et al., 2009); 
MMW fulfills both criteria for meaningful PBL. The scenarios 
developed and included in the application present students 
with real issues and ask them to develop strategies, based on 
what they have learned, and apply those strategies to 
develop plans to address them. What makes this approach 
such an effective way to engage students is the emphasis on 
solving real problems as a team in a collaborative setting; 
students participate in every step of the process by: (1) 
defining the key challenges of the problem; (2) deciding who 
the stakeholders are and what issues affect them; and (3) 
analyzing data and discussing potential solutions. This 
makes the science engaging and relevant to real life. It also 
models the interdisciplinary approach that environmental 
decision making takes in professional settings while 
developing communication and collaboration skills. 

The National Center for Case Study Teaching in Science 
(NCCSTS, 2013) has compiled an impressive set of examples 
of how this format can be adapted to science education at 
every level. The scenarios for MMW are developed as 
dilemma/decision cases, as defined by NCCSTS (http:// 
sciencecases.lib.buffalo.edu/cs/collection/). 

MMW SCENARIOS 

MMW scenarios are incorporated into the application 
through the teacher portal. These problems are situated in a 
range of settings from urban through suburban to rural 
areas. In each case, users are asked to seek solutions that not 
only address the stated problems, but are also tasked to 
forge a solution that considers cultural, social, economic, and 
social-justice issues. Next, we present a summary of 
scenarios that are currently included in the MMW applica¬ 
tion. 

• The Mill Creek neighborhood is located in West 
Philadelphia, PA. When William Penn first designed 
the street layout for his "Greene Country Towne," 
between the Delaware and Schuylkill Rivers, he 
envisioned a grid pattern with a green square at the 
center and one in each quadrant. In the early 19th 
century, Philadelphia expanded into the agricultural 
area west of the Schuylkill River. This area included 
streams, many of which powered local mills. Rather 
than considering the natural topography and hydrol¬ 
ogy, the city planners simply imposed the existing grid 
pattern over the landscape. Streams, like Mill Creek, 
were originally used as open sewers but were 
eventually buried in pipes and became part of 
Philadelphia's sewer network. As do so many buried 
streams, Mill Creek did not stay confined in its pipe. 
In the mid-1900s, there were two dramatic collapses 
that undermined houses and even swallowed a car. 
The decline of the buildings mirrored the decline in 
the socioeconomic status of the neighborhood's 
residents. Although the creek was not the only cause 
of neighborhood decline, it did exacerbate the 
situation. Despite redevelopment during the 1960s, 
the neighborhood continues to be economically 
depressed and subject to periodic collapses (Spirn, 
2005). The MMW scenario asks students to redevelop 
the neighborhood around a park that represents the 
former channel of the stream. They are advised to 
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incorporate best land management practices such as 
tree plantings and infiltration basins within open 
space; to provide commercial and employment 
centers; and to build sufficient housing to provide 
for all current residents. Their plan must reduce 
storm-water runoff that causes Philadelphia's com¬ 
bined sewers to dump raw sewage into the Delaware 
and Schuylkill Rivers (Philadelphia's water-supply 
sources) during even moderate rainstorms. They must 
also maintain community integrity and provide 
needed local resources. 

• Valley Forge National Historic Park is located at the 
confluence of the Valley Creek and the Schuylkill 
River. In the winter of 1777, General George 
Washington and nearly 3,000 troops wintered at 
Valley Forge, Pennsylvania, 20 miles (32 km) north 
of Philadelphia. At the time, the land was actively 
farmed. Today, the area surrounding Washington's 
Headquarters is part of Valley Forge National 
Historical Park, which maintains much of the rural 
character it had during Washington's encampment. 
However, the headwaters of Valley Creek have been 
developed with condominiums and apartment devel¬ 
opments that cause increased storm-water runoff and 
regularly cause flash flooding. Even moderate rainfall 
causes the National Park Service personnel to place 
sandbags around Washington's Headquarters to 
prevent damage to the structure. Specifically, this 
scenario asks students to outline a plan to: (1) reduce 
the amount of overland flow that reaches the creek 
during storm events by implementing best-manage¬ 
ment practices and (2) stabilize downstream, flood- 
prone areas such as Washington's Headquarters using 
riparian plantings. In their plan, students must not 
only protect George Washington's Headquarters, they 
must also maintain the integrity of the headwater 
community (number of households, commercial and 
employment centers) to minimize the impact to 
current residents. 

• Lancaster County, PA, is known for its rich, limestone 
soils that support the agriculture of the Plain Sect 
(Amish and Mennonite) farmers. Despite the tourist 
industry that has grown around the picturesque 
Amish farmsteads and buggies, the Plain Sect 
residents are an endangered culture. Suburban sprawl 
threatens their productive farmland, which could soon 
sprout houses rather than corn. In this scenario, users 
are asked to preserve prime farmland to protect the 
Plain Sect culture. The users compare four local land 
areas: (1) the farmland surrounding their schoolyard; 
(2) a recently constructed housing development; (3) 
an urban marketplace; and (4) a forested area. Using 
best-management practices, users model ways to best 
manage community growth. 

In all of these scenarios, students are asked to justify 
their decisions and to discuss how they are addressing local 
societal and economic issues in a culturally sensitive way. 
Depending on the grade level, they may be asked to discuss 
some or all of the following questions as they prepare their 
plans. The level of complexity and detail expected from 
students will, of course, vary with grade level. 
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1. Who are the local stakeholders? Students are asked to 
identify all of the groups who are affected by the 
problem or who will be affected by the proposed 
solutions. They also need to identify economically 
and culturally vulnerable populations. For those 
stakeholders who may not be actively engaged in 
local decision making (e.g., Plain Sect farmers or 
underrepresented minorities), they should also de¬ 
fine a strategy for incorporating them into the 
planning process. This could include Web-based 
research on how these populations have responded 
to similar problems. It could also include face-to-face 
interviews with group representatives, if possible. 

2. What are each stakeholder's concems/issues? Once the 
student or team has identified all of the stakeholders, 
they should then outline any stakeholders' concerns 
or issues that would relate to any proposed solution. 
Once they have done that, they should identify areas 
of potential conflict that will need to be addressed. 
Each stakeholder concern should be categorized. For 
example, is the concern economic, cultural, religious, 
ideological, political, etc. This will be important to 
identifying those concerns on which stakeholders 
may be able to compromise and those where little or 
no compromise is possible. 

3. Where are there direct conflicts and where can trade-offs 
be accommodated easily? Students are asked to identify 
areas where they believe compromise is possible and 
where trade-offs can address conflicting viewpoints. 
For example, would one group be willing to accept 
the proposed changes in exchange for walking and 
bicycle trails? 

4. Where there are no easy compromises, how should 
conflicting viewpoints and interests be resolved? Stu¬ 
dents should consider how local land-use decisions 
get made and how they should be made. Would they 
propose a public meeting? If so, how would that be 
managed? Is there another alternative way to make 
decisions? 

5. Ultimately, how should decisions get made and by 
whom? Students should discuss how difficult envi¬ 
ronmental decisions should be made in a democratic 
society and how minority interests should be 
addressed. 


RESEARCH ON EFFECTIVE WATERSHED 
INSTRUCTION 

Improving watershed instruction is not as simple as 
merely implementing compelling scenarios into the learning 
process. Designing effective instruction must build upon 
students' current understanding of water. Students' initial 
ideas about water and watershed topics are often naive and 
unconnected, but there is hope for change. Gunkel et al. 
(2012), Endreny (2010), and Ben-Zvi Assaraf and Orion 
(2005a, 2005b, 2010) provide evidence that children have the 
capacity to develop more connected, sophisticated, and 
systems-oriented ideas about water through instruction. 
However, compelling evidence suggests that students' 
conceptual understanding of scientific topics, including 
environmental topics, bears little impact on the actual 
decisions they make on real-world issues (Allum et al., 
2008; Nisbet and Scheufele, 2009). Similarly, in classroom 
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settings, students typically make few connections between 
the content and their decisions (Zohar and Nemet, 2002; 
Sadler, 2004; Raved and Ben-Zvi Assaraf, 2011; Rose and 
Barton, 2012). To improve watershed understanding, curric¬ 
ulum must be purposefully designed to provide students 
meaningful opportunities to connect conceptual under¬ 
standing of watersheds to real-world decision making. The 
National Research Council (NRC) Framework emphasizes 
scientific practices including developing and using models, 
analyzing and interpreting data, engaging in arguments 
from evidence, and obtaining, evaluating, and communicat¬ 
ing information. The MMW project is based on the premise 
that this barrier between students' conceptual understanding 
of watershed content and their real-world actions can be 
bridged by the enactment of a curriculum that teaches 
watershed content using real data, scientific models, and 
local places. The MMW project allows students to actively 
explore their own watershed. To assess the effectiveness of 
the MMW curriculum, a research study was conducted to 
investigate the following questions. 

1. What is the impact of the MMW curriculum on 
students' conceptions of the watershed? 

2. Are students able to apply watershed content learned 
through the MMW curriculum to real-world prob¬ 
lems? 

3. What misconceptions about the watershed persist 
after instruction? 

Research Design 

An investigation of the Lancaster County, PA, scenario 
was piloted in the fall of 2012 with two secondary Earth 
Science teachers, six classrooms, and approximately 150 
students in a rural Lancaster County High School. In this 
pilot, each teacher used the MMW curriculum with two of 
their classes and their traditional curriculum with one of 
their classes. This pilot study suggested that students' 
struggled to connect the concepts of runoff and land 
management to the larger water cycle. Still, students 
reported enjoying using the MMW portal, but they often 
missed potential learning opportunities available in the 
application, suggesting that the curriculum needed increased 
scaffolding to lead students through the application. The 
results of the pilot were used to refine the MMW curriculum 
and analysis instruments. A full-scale study was then 
conducted in the spring of 2013. Four teachers from two 
rural school districts in Lancaster County, two 7th grade life 
science teachers, and two high school environmental science 
teachers implemented the MMW curriculum in 16 class¬ 
rooms. The MMW curriculum was designed to address local 
issues and was adapted for the age of the students. The 7th 
grade life science teachers chose to investigate a 1 square 
mile area (1.6 km 2 ) surrounding their rural school campus. 
Using a carefully scaffold curriculum that guided students 
through the Simple Model and Show My Watershed 
applications, students assessed the watershed health of their 
"school yard." The students then compared the schoolyard 
environment to a 1 square mile area (1.6 km 2 ) surrounding 
the "city market" in downtown Lancaster, PA. The high 
school classrooms utilized a similar scaffold curriculum that 
expanded the investigation to include five areas in Lancaster 
County: the schoolyard, a housing development, a local 
farm, a forested area, and the city market. In the high school 


curriculum, students were guided through the Simple 
Model, Show My Watershed, and Modify My Watershed. 
Model My Watershed Curricula developed by various 
teachers who used MMW are available at http:// 
wikiwatershe d. org/curricula. html. 

Research Instruments 

The study used a mixed-method design that involved 
the simultaneous collection and analysis of both quantitative 
and qualitative data. The instruments used in this study 
included: a 14 question content knowledge test, the Draw 
My Watershed (Shepardson et al., 2007) assessment, and 
semistructured interview protocol for focus groups. 

The content knowledge test was developed by the 
research team, which included watershed scientists, science 
education researchers, and project evaluators. The content 
knowledge test was administered online to students before 
and after instruction. The content knowledge questions were 
integrated into a larger survey that included questions about 
interest in science, technology, engineering, and mathemat¬ 
ics (STEM) and future career interests. The "Draw My 
Watershed" assessment was created by Shepardson et al. 
(2007). This assessment asks students to draw a picture of a 
watershed and to describe their drawing in a paragraph 
response. This assessment is scored using a study-designed 
rubric that assesses students' watershed understanding 
according to the scale developed by Gunkel et al. (2012). 

All students in the study completed a pre- and postunit 
quantitative/qualitative test of watershed content knowledge 
and a pre- and postunit qualitative "Draw My Watershed" 
(Shepardson et al., 2007) assessment. After the completion 
of the unit, each of the four teachers in the study chose six to 
eight students to participate in a focus group using a 
semistructured interview protocol designed by the research 
staff. 


RESULTS 

Data collected from the quantitative portion of the pre/ 
postunit content knowledge survey show a statistically 
significant gain in content knowledge for students in all 
classes (Table I). 

Pre- and post-test data were collected from the Draw 
My Watershed Assessment (Table II). Using the four 
categories of watershed understanding outlined by Gunkel 
et al. (2012) as a guide, the Draw My Watershed Assessment 
(Shepardson et al., 2007) data were analyzed with a rubric 
that rated students' watershed understanding. The rubric 
consisted of a five-point scale, where 0 = no/incorrect 
understanding and 4 = comprehensive understanding. 
Examples of answers for each level appear in Figure 4. 

All classes showed statistically significant improvement 
in understanding, yet few achieved the highest levels of 
understanding. 

Data from the qualitative survey item and an analysis of 
the Draw My Watershed instrument show that students' 
understanding of watersheds increased, especially in their 
understanding of large-scale connections of water, i.e., 
streams to rivers. However, students were still lacking in 
their understanding of microconnections related to the 
"hidden sources" of water, e.g., groundwater and water 
vapor. In addition, while many students recognized many 


J. Geosci. Educ. 62, 61-73 (2014) 


Model My Watershed: A Place-Based Web Application 67 


TABLE I: Data collection. 


Research Question 

Instrument 

Data Type 

1. What is the impact of the MMW 
curriculum on students' conceptions of 
the watershed? 

14 question survey that investigates students' 
watershed content knowledge and their ability to 
apply this knowledge to real-world problems (pre/ 
post). 

Quantitative and qualitative question 

2. Are students able to apply watershed 
content learned through the MMW 
curriculum to real-world problems? 

Qualitative questions from content survey (pre/ 
post) and student focus group (postinstruction 
only). 

Qualitative 

3. What misconceptions about the 
watershed persist after instruction? 

Draw My Watershed assessment (Shepardson et 
al., 2007). This instrument asks students to draw a 
picture of their watershed and to explain in words 
what a watershed is (pre/post). 

Qualitative and quantitative 


TABLE II: Pre/postcontent knowledge survey. 


Teacher 

Pretest 

Post-Test 

Difference 

f-Test 1 

All (n = 270) 

5.1 (2.5) 

8.5 (2.1) 

3.3 

19.8** 

7th grade teacher A (n = 112) 

4.4 (2.1) 

7.1 (2.9) 

2.7 

10.8** 

7th grade teacher B (n = 59) 

5.1 (2.5) 

8.5 (2.3) 

3.4 

y y** 

High school teacher A (n = 44) 

4.7 (2.1) 

9.1 (2.1) 

4.4 

11.0** 

High school teacher B (n = 55) 

6.8 (2.3) 

11.1 (1.4) 

4.3 

13.3** 


*Mean (standard deviation) difference between pre- and postinstruction and t-test (** = p < 0.01). 


TABLE III: Draw My Watershed analysis. 


Teacher 

Number of 

Matched 

Pairs 

Pre 

Post 

f-Test 1 

7th grade teacher A 

74 

0.9 (0.8) 

1.3 (0.9) 

2.8** 

7th grade teacher B 

85 

0.4 (0.7) 

1.1 (0.7) 

6.9** 

High school teacher A 

80 

0.7 (0.7) 

2.0 (1.2) 

8.4** 

High school teacher A 

44 

0.8 (0.7) 

1.8 (0.6) 

7.1** 

Overall 

283 

0.7 (0.8) 

1.5 (1.0) 

11.8** 


Tdean (standard deviation) difference between pre- and postinstruction and f-test (** = p < 0.01). 


water pathways, they usually failed to identify when one 
pathway would be more likely than another. 

Four focus groups were conducted at the conclusion of 
the unit. Each of the teachers involved in the study 
assembled a purposeful selection of five to seven students 
to participate in a 45 min focus group. Students were chosen 
in order to have a range of ability levels, interest in science, 
and gender balance. Using a semistructured interview 
protocol (see Appendix A), students provided information 
on their experience using the Model My Watershed toolset. 
In the pilot study, students expressed concern that the 
classroom instruction did not support their use of the Model 
My Watershed toolset, and they saw a disconnect between 
the Model My Watershed maps and their local area. This 
information was used to create a scaffold approach for the 
curriculum used in the full-scale study, where the MMW 
toolset and classroom instruction were fully integrated 
through the creation of an activity guide for classroom use 
(http://wikiwatershed.org/curricula.html). In the full-scale 
study, students did not express similar concerns or curricular 
disconnect. The analyses of data from the four focus groups 


conducted in the full-scale study were carried through in 
several steps using NVivo to organize the developing 
different themes and identify critical incidents (Braun and 
Clarke, 2006). The analysis resulted in the identification of 
three themes as described in Table IV. Generally, the results 
highlighted that the use of technology allowed for a 
personalization of learning, where each student could use 
his or her own yard as the context for watershed study, and 
the MMW toolset allowed for quick and varied experimen¬ 
tation of differing environmental conditions. This ability to 
quickly test the impact of best-management practices on the 
local watershed was cited by students in their explanation of 
how best-management practices, tested in the MMW 
toolset, could positively impact the health of their watershed. 
Finally, when asked for suggestions on how to improve the 
MMW toolset, unlike during the pilot study, students did not 
offer substantive suggestions related to curriculum align¬ 
ment or connections. Instead, students' suggestions were 
restricted to design features. Particularly, students warned 
against the use of text-rich descriptions or directions by 
directly stating that they do not read the text and that they 
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1. Explain the drawing in your own words. Write at least 2 sentences. 
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A watershed is all of the land where water from precipitation drains to a single stream, river, 
lake or ocean Water that fans as rain or snow (once melted), either infiltrates into the 
ground or. if the surface is impervious (paved, compacted or frozen), flows overland to 
reach downhill water bodies Water that infiltrates moves under ground and reaches 
streams and rivers, contnbuting to base-flow Vegetation transpires water back lo the 
atmosphere as a result of photosynthesis Water also evaporates from surfaces, 
contributing to relative humidity in the atmosphere Once the atmosphere is saturated, if 
there are sufficient condensation nuclei, it rains In developed areas, there are often paved 
or other hard surfaces, such as roofs, where water cannot infiltrate That water runs off and 
enters storm sewers In older cities with combined storm water and waste-water sewer 
systems, an influx of runoff can cause those sewers to discharge raw sewage into local 
rivers or lakes 


FIGURE 4: Sample drawings showing different levels of watershed comprehension. Level 4 was completed by a 
member of the research team, (a) Level Zero Example. Water is depicted as a literal shed or other incorrect/off-base 
answer. Answer shows cartoonish characterization with no connection to environmental features, (b) Level One 
Example. Water is visible in familiar, isolated contexts, such as rivers, puddles, lakes, oceans, faucets, or isolated 
water. Answer does not acknowledge hidden sources of water, such as groundwater or water vapor. Map pictures 
show no connection between water features, (c) Level Two Example. Answer includes some mechanisms to move 
water. However, human actions to move water are not stressed. Connections to hidden sources of water, e.g., 
groundwater and water vapor, are not stressed. Large-scale connections of water, i.e., streams to rivers are shown but 
microconnections, such as groundwater recharge and contribution to stream base flow, are missing, (d) Level Three 
Example. Answer includes all features listed above for Level 2 and includes connections to school science. Illustration 
traces water among many pathways but does not indicate the likelihood of one pathway over another or the reasons 
for those differences, (e) Level Four Example. Integrates connections in the water cycle, including when and where 
water will follow specific pathways, such as infiltration, evaporation, transpiration, and runoff. Links these pathways 
to human influences and activities in the watershed, e.g., runoff from impervious surfaces as opposed to infiltration. 


find pages containing a lot of text to be boring and 
unengaging. 

The data provide evidence that the MMW Web site and 
curricula are effective tools for increasing students' under¬ 
standing of watersheds and the impact of human decisions 
on local watershed health. By using the students' local 
watershed as the location of exploration and learning, the 
MMW project connects "school science" content to real- 
world applications by using scientific data, knowledge, and 
practices to participate in evidence-based decision making 


about issues impacting their local watershed (Kali et al., 
2003; Orion and Ault, 2007; Mohan et al., 2009; Gunkel et 
al., 2012). While statistically significant learning gains were 
measured, most students failed to reach the highest levels of 
watershed understanding as defined by Gunkel et al. (2012). 
Further refinement of the MMW curricula is needed to help 
students trace water among many pathways and to be able 
to predict the likelihood that water will follow one pathway 
over another and the connection between "hidden" sources 
of water (groundwater and water vapor). 
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TABLE IV: Focus group analysis. 


Interview Question 

Emerging Themes 

Selected Quotes from Responses 

I am wondering if you can tell me what you have 
learned about watersheds and water quality since 
the beginning of this unit a couple of weeks ago? 
What were the most surprising or interesting things 
you learned about? 

Role of technology in 
the learning process 

I thought it was helpful because it wasn't on paper so 
we could explore it and go online and click on 
different things to see how it changes the environment 
so you could see the changes. I thought that was 
good. 

They thought it was cool seeing their own house. 

"There you go, there's my own house!" 

I wonder if any of you plan to do anything 
differently since learning about how to keep 
watersheds healthy in this unit? Are there realistic 
things you could do (or you could encourage others 
to do) to keep your watershed healthy? 

Actions students can 
take 

So like, before when we are learning about change 
how if you want a different amount of runoff. 

Sometimes it's hard to think about how you could 
change it. But some of it is clear and its right there. 

You can plant more green things, and you can have a 
green roof. 

Taking care when you build houses and creating green 
space. 

Did any of you visit the WikiWatershed Web site 
on your own time (e.g., NOT homework)? Did any 
of you explore it further? If so, what did you think? 
[For MMW students:] Did you talk about 
WikiWatershed with friends who were not in this 
class with you? If so, what did they think? 

Suggestions for 
improvements to the 
MMW interface 

They don't read directions. Too many "words" gives 
the impression of "boring" 

I didn't read it—it just looks boring. 


SUMMARY 

MMW provides students with the tools necessary to 
investigate their local watershed with a focus on their role as 
agents of hydrologic change. The application is designed 
using problem-based learning principles implemented as a 
series of proposed scenarios that each present a difficult 
environmental problem where there are many conflicting 
interests. MMW provides a vehicle where environmental 
issues can be addressed within local social, economic, and 
political contexts. This is an essential curriculum design 
principle, as most environmental challenges are complex 
and imbued with issues of potential social and economic 
conflict. In short, there are no simple answers, but there may 
still be workable solutions to the issues that we face. 
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Appendix 

Pre/Post Content Knowledge Survey Questions 
Knowledge Test and Answer Key 

Q15 If you were responsible for planning a new area for housing, which is the best building design to provide everyone 
with a place to live while protecting the environment? 



O Apartment with Green Roof (2) 




Q19 A watershed is: 

0— the land next to a stream or river that often floods after a large rain storm (1) 

0— a building built over a spring where people collect water (2) 

O the land area where all rain drains into a single river, lake or stream (3) 
0— the process of water running down hill towards a stream, river or lake (1) 

Q28 True or false? Cluster housing is way to limit watershed impact in a populated area. 

O True (1) 


0 False (2) 



Q30 



Using the figure above, match a number to the corresponding part of the water cycle. 


_2_ Precipitation (1) 

_5_ Evaporation (2) 

_3_ Infiltration (3) 
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Q31 Which of the following is NOT a “best-management practice”? 


0— porous paving (2) 

O storm drain (4) 

Q32 True or False? Water drains slower through sandy soil than through soil with lots of clay. 
-0-— True (1) 

O False (2) 


Q33 Trees and other plants take water from the soil and release it into the atmosphere. What is the name of that 
process? 


O Transpiration (3) 


Q34 Turf grass lawns have more runoff than natural grass meadows because . . . 


0— their root structure is so dense (1) 

0— the underlying soil has been compacted by repeated mowing (2) 

O both of the above (3) 

0— none of the above ( 4 ) 


Q35 Fill in the blank. 

_Hydrology_is the study of water and its movement on and through the earth. 

Q36 Fill in the blank. 

_Sediment_are particles of rock and soil that are washed into a stream or river. 


Q37 Which of the following land uses might cause the least amount of storm water runoff? 
0— a baseball field (1) 


O a forest (3) 


Q39 Meteorologists predict the weather by using models. Models allow scientists to understand how processes work. 
Which of the statements is NOT a true statement about models? 

0— Models are representations of real objects or processes. (1) 

O Models are 100% accurate. (2) 

0— Models use large amounts of data to make predictions of real systems. (3) 

0— Models simulate large - scale experiments that would be too costly or impossible to actually perform. ( 4 ) 
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Q40 Briefly describe, in steps, how a raindrop that lands on your backyard could make its way to the ocean. 

_A raindrop would runoff of the lawn and flow to a storm drain then through the storm drains to a nearby 
river, or it could runoff directly into a stream. It would then follow the path of the stream directly into a creek, 
then into a river and eventually into the ocean_ 

Q41 



Look at the above figure. What best management practices could be added to 
this house or yard that would minimize storm-water runoff? 

1) Table with umbrella, solar panels and livestock 

2) Rain gardens, porous pavers and rain barrel 

3) Wind turbine, green roof and submerged cistern 

4) Solar panel, rain barrel and bird bath 

5) I don't know (5) 

Student Focus Group Questions: 

Introduction: 

As you might remember from when we were first introduced, we’re trying to learn how a new curriculum works. 
Some of you were involved in a new, experimental learning software and some students learned the same content 
through a more traditional teaching strategy. So you might have heard that students in another class were doing 
something quite different. That’s OK and we would like to hear about both experiences if you heard about it outside 
of class. 

1. Did any of you visit the WikiWatershed website on your own time (e.g. NOT homework)? Did any of you 
explore it further? If so, what did you think? [For MMW students:] Did you talk about WikiWatershed with 
friends who were not in this class with you? If so, what did they think? 

2. Iam wondering if you can tell me what you have learned about watersheds and water quality since the 
beginning of this unit a couple of weeks ago? What were the most surprising or interesting things you 
learned about? 

3. Next, I’m curious about how you feel about science? {Note: This is different than what you know about 
science or watersheds. How you feel can include how interested you are in science as well as other 
feelings you may have.) If you can remember this far back, can you tell me a little about how you felt about 
science before this unit? Do you think your feelings have changed at all? Why? 

4. What are your feelings about how water is managed in your community? 

5. How many of you think you might want to have a science job when you get older? How do you imagine 
yourself using science (maybe in the ways you learned from the watershed course) in your career or just 
being a part of a community when you are done with your education? What do you imagine yourself doing 
for your career? 

6. Finally, I wonder if any of you plan to do anything differently since learning about how to keep watersheds 
healthy in this unit? Are there realistic things you could do (or you could encourage others to do) to keep 
your watershed healthy? 






























